@A Method of Partly Automated Testing of Software 

Principles of symbolic execution and temporal monitoring are exploited. 

Ames Research Center, Moffett Field, California 


Property-Generator 
™ Module 

T 

Test-Input- 

Generator ► Program Under Test 

Module 


Instrumentation 

Module 


Four Software Modules are used together to determine (1) what properties the program under test 
should have and (2) whether it does, indeed, have those properties. 
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A method of automated testing of soft- 
ware has been developed that provides 
an alternative to the conventional mostly 
manual approach for software testing. 
The method combines (1) automated 
generation of test cases on the basis of 
systematic exploration of the input do- 
main of the software to be tested with 
(2) run-time analysis in which execution 
traces are monitored, verified against 
temporal-logic specificadons, and ana- 
lyzed by concurrency-error-detection al- 
gorithms. In this new method, the user 
only needs to provide the temporal logic 
specifications against which the software 
will be tested and the abstract descrip- 
tion of the input domain. 

For testing a given computer program, 
the method involves the software analog 
of a hardware test harness, consisting of 
four software modules: a test-input-gener- 
ator module, a property-generator mod- 
ule, a program-instrumentation module, 
and an observer module (see figure). 
The test-input-generator module auto- 
matically generates inputs to the pro- 
gram under test, one at a time, on the 
basis of a previously developed symbolic- 
execution approach. In this approach, 
symbolic values (instead of data) are used 
to represent program values and the state 
of a symbolically executed program is 
represented by a combination of (1) the 
symbolic values, (2) a program counter, 
and (3) a path condition in the form of a 
Boolean formula over the symbolic in- 
puts that accumulate constraints that the 
inputs must satisfy in order to make exe- 
cution follow a particular associated path. 


An input generated by the test-input- 
generator module is fed to the property- 
generator module, which automatically 
generates a set of properties that the 
program under test is required to ex- 
hibit when executed in response to the 
given input. The input is then fed to the 
program under test. The program is 
then executed and generates an execu- 
tion trace. 

The instrumentation and observer 
modules perform the aforementioned 
run-time analysis. The instrumentation 
module must be constructed to report 
events that are relevant for determining 
whether the program exhibits the re- 
quired properties during a particular ex- 
ecution. The observer accepts, as input, 
the execution trace and the set of proper- 
ties generated by the property-generator 
module to determine whether the pro- 
gram exhibits the required properties. 


The test-input-generator and prop- 
erty-generator modules must be con- 
structed specifically for the program 
under test. It may be possible to auto- 
mate the construction of the instrumen- 
tation module, depending on the nature 
of the program under test. The observer 
module is generic and can be re-used for 
testing other programs. 
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